Focusing on the issue that the Correlation Filter (CF) has poor performance in tracking fast motion object, a Long-term Kernelized Correlation Filter (LKCF) tracking algorithm based on optical flow combining with Kernel Correlation Filter (KCF) was proposed. Firstly, while tracking with the tracker, a value of Peak-to-Sidelobe Ratio (PSR) was calculated. Secondly, the position was achieved in the last frame, optical flow was used to calculate coarse position when the value of PSR less than a threshold in the current frame, which means tracking failure. Finally, accurate position was calculated using the tracker again according to the coarse position. The results of experiment compared with four kinds of tracking algorithms such as Compressive Tracking (CT), Tracking-Learning-Detection (TLD), KCF and Spatio-Temporal Context (STC) show that the proposed algorithm is optimal in distance accuracy and success rate which are 6.2 percentage points and 5.1 percentage points higher than those of KCF. In other words, the proposed algorithm is robust to the tracking of fast motion object.